Skip to content

Conversation

j9liu
Copy link
Contributor

@j9liu j9liu commented May 29, 2025

Description

Depends on CesiumGS/cesium-native#1188.
Depends on #1694, so merge that first.

This PR adds support for loading and rendering tilesets with 3DTILES_content_voxels, parsing glTFs with EXT_primitive_voxels payloads.

image

Data courtesy of Swisstopo

This is a meaty set of changes, so I'll add a written walkthrough of the changes in a follow-up comment.

Author checklist

  • I have submitted a Contributor License Agreement (only needed once).
  • I have done a full self-review of my code.
  • I have updated CHANGES.md with a short summary of my change (for user-facing changes).
    - [ ] I have added or updated unit tests to ensure consistent code coverage as necessary.
  • I have updated the documentation as necessary.

Remaining Tasks

  • Improve memory / lifetime management
  • Revisit voxel traversal mechanics. -- See changes in Add support for ellipsoid-based voxels #1711
    • I'm not happy with the way voxels are currently selected for rendering in VoxelResources. It's a weird compromise between CesiumJS's VoxelTraversal and the cesium-native traversal, and it falls apart for larger datasets.

Testing plan

Reach out to me offline for the pictured test data.

  • Load in a voxel tileset (same as any other Cesium3DTileset).
  • Use UCesiumVoxelMetadataComponent to gather its properties.
  • Test out applying custom shaders to the data.

Compatibility checks:

Unreal Version DX 11/12 Vulkan Metal Editor Built
5.4
5.5
5.6

@j9liu

This comment was marked as outdated.

@j9liu j9liu changed the base branch from main to property-attributes June 11, 2025 18:09
@j9liu j9liu changed the base branch from property-attributes to main June 17, 2025 19:54
@j9liu j9liu changed the base branch from main to property-attributes June 17, 2025 19:55
@j9liu
Copy link
Contributor Author

j9liu commented Jul 3, 2025

I could keep picking apart at this, but I think this would benefit more from a fresh pair of eyes. I'm keeping this as a draft but I would appreciate any and all reviews 🙏

@j9liu
Copy link
Contributor Author

j9liu commented Jul 9, 2025

Turns out that UE 5.4 has drastically improved the UX of Custom HLSL nodes in materials!

It's now possible to define extra #include paths in for the shader, so I don't need to use one uber shader anymore. I've reorganized the code so that everything is separated into meaningful pieces. Hopefully this makes things easier to review, too ^^

@j9liu j9liu marked this pull request as ready for review July 10, 2025 20:41
Base automatically changed from property-attributes to main July 30, 2025 23:03
@kring
Copy link
Member

kring commented Oct 16, 2025

I was able to get a basic thing working:
image

There were a few things I didn't realize, and they took me awhile to figure out:

  • The voxel material layer must be the bottommost layer in the material. I initially thought, for some reason, that it had to be layered on top of a ML_CesiumGltf like our other materials. And when that didn't work, I assumed it was because I misnamed the material layer. Only after trying and failing to find the expected material layer name in the code did I realize it was hard-coded to layer 0.
  • The shape of this dataset isn't apparent unless your material sets the alpha to 0.0 when Temp_C == Temp_C_NO_DATA. I initially tried to just return a solid color and wondered why all I got was a rectangle.
  • But even after the above, I still got a rectangle. Because my material had a Blend Mode of "Masked" instead of "Translucent".

So a lot of this wouldn't have been a problem if we had a tutorial. But if the generated material layer isn't really meant to be used in a layered material, perhaps we should just generate a material directly? That would have eliminated two out of my three errors above.

Also, I'm wondering if the "Voxel Raymarch" custom expression could have an output node for each of the CustomShaderProperties? The way the styling of other types of tiles works. Then users could implement their logic using the shader graph. Also, it seems like pressing the "Auto Generate" button again clobbers any edits to that custom expression node, so the chance of losing the user's work is pretty high with the way things are set up now.

@j9liu
Copy link
Contributor Author

j9liu commented Oct 16, 2025

Thanks for the feedback @kring ! I agree with your points about setting up the material and generating it as a full material instead of as a layer.

Also, I'm wondering if the "Voxel Raymarch" custom expression could have an output node for each of the CustomShaderProperties? [...] pressing the "Auto Generate" button again clobbers any edits to that custom expression node, so the chance of losing the user's work is pretty high...

Yeah, I acknowledge this system isn't great. The problem is that raymarching must be done in a for loop to actually find valid voxels in the first place, and I don't see how to enable that from material nodes. I wish there was a way separate the raymarch logic from the node-based styling, I don't know how you'd re-call the raymarching logic (and not only that, to stop it once it's reached an efficiently saturated output). I can experiment with this some more, but I'm not super hopeful I'll find a better solution...

@j9liu
Copy link
Contributor Author

j9liu commented Oct 16, 2025

Tried hacking a for-loop by using a couple custom nodes in a material, but it looks like Unreal doesn't like it:

LogMaterial: Warning: [AssetLog] /Engine/Transient.Material_0: Failed to compile Material for platform PCD3D_SM5, Default Material will be used in game.
    Expression is part of a cycle. Please make sure the material graph is acyclic.

Makes sense though... it'd probably end up nesting calls like functionA(functionB(functionC(functionA... etc.

@kring
Copy link
Member

kring commented Oct 17, 2025

Works well on macOS / Metal (UE 5.5):
image

@kring
Copy link
Member

kring commented Oct 20, 2025

Here are some initial review comments. There's a lot here and I haven't been through everything yet. Overall it looks great, though!

  • I noticed that the tileset disappears when I get too close to it (inside its bounding volume, presumably). How realistic is it to avoid that? I was hoping to be able to fly through the voxel field. 😀

  • I can get this weird lighting effect just by rotating the camera. I say it's lighting related because it goes away in the "Unlit" view. Any idea what this is?
    lighting-flash

This is my (not particularly well considered) shade function:

return Properties.Temp_C_NO_DATA == Properties.Temp_C ? float4(0.0, 0.0, 0.0, 0.0) : lerp(float4(0.0, 1.0, 0.0, 0.02), float4(0.0, 0.0, 1.0, 0.02), (Properties.Temp_C - 10.0) / 100.0);

{
%s

float4 Shade(CustomShaderProperties Properties)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not make this Shade function a member of CustomShaderProperties instead? Then the properties can be accessed without qualifying them with Properties..

(TitleProperty = "Custom Shader",
DisplayAfter = "CustomShaderPreview",
MultiLine = true))
FString CustomShader = TEXT("return 1;");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest changing the default value to something like return float4(1.0, 1.0, 1.0, 0.02);. That makes it much clearer that this is a color, and it makes the default visualization look at a bit more voxely.

FString CustomShader = TEXT("return 1;");

/**
* Any additional functions to include for use in the custom shader.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Any additional functions to include for use in the custom shader.
* Any additional functions to include for use in the custom shader. The HLSL code provided here is included verbatim in the generated material.

TCHAR_TO_UTF8(*dbFile));
#endif

this->_pTileset->getRootTileAvailableEvent().thenImmediately([thiz = this]() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic looks like it needs to run in the main thread, so thenImmediately is dangerous.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, thiz = this is pointless because thiz has the same type as this. You might as well just capture this.

There's a bit of danger here though in that the Cesium3DTileset could get garbage collected before the root tile is available. And I don't think there's anything that would prevent the root tile available event from still being raised. I think a simple IsValid(this) - and early return - should be sufficient to guard against that. We do the same thing in SampleHeightMostDetailed.

this->_pVoxelRendererComponent->UpdateTiles(
pResult->tilesToRenderThisFrame,
pResult->tileScreenSpaceErrorThisFrame);
} else {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A single tileset could have both voxel and non-voxel content, right? In that cause this else would be wrong, I think.

Comment on lines +2395 to +2396
const FCesiumVoxelClassDescription* pVoxelClassDescription =
this->_voxelClassDescription ? &(*this->_voxelClassDescription) : nullptr;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear to me why this _voxelClassDescription is squirreled away earlier, and then accessed here. Why not get the description directly from the voxel metadata component here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants